Using machine learning to improve the accuracy of patient deterioration predictions: Mayo Clinic Early Warning Score (MC-EWS).

OBJECTIVE: We aimed to develop a model for accurate prediction of general care inpatient deterioration. MATERIALS AND METHODS: Training and internal validation datasets were built using 2-year data from a quaternary hospital in the Midwest. Model training used gradient boosting and feature engineering (clinically relevant interactions, time-series information) to predict general care inpatient deterioration (resuscitation call, intensive care unit transfer, or rapid response team call) in 24 hours. Data from a tertiary care hospital in the Southwest were used for external validation. C-statistic, sensitivity, positive predictive value, and alert rate were calculated for different cutoffs and compared with the National Early Warning Score. Sensitivity analysis evaluated prediction of intensive care unit transfer or resuscitation call. RESULTS: Training, internal validation, and external validation datasets included 24 500, 25 784 and 53 956 hospitalizations, respectively. The Mayo Clinic Early Warning Score (MC-EWS) demonstrated excellent discrimination in both the internal and external validation datasets (C-statistic = 0.913, 0.937, respectively), and results were consistent in the sensitivity analysis (C-statistic = 0.932 in external validation). At a sensitivity of 73%, MC-EWS would generate 0.7 alerts per day per 10 patients, 45% less than the National Early Warning Score. DISCUSSION: Low alert rates are important for implementation of an alert system. Other early warning scores developed for the general care ward have achieved lower discrimination overall compared with MC-EWS, likely because MC-EWS includes both nursing assessments and extensive feature engineering. CONCLUSIONS: MC-EWS achieved superior prediction of general care inpatient deterioration using sophisticated feature engineering and a machine learning approach, reducing alert rate.

The fifth vital sign? Nurse worry predicts inpatient deterioration within 24 hours.

INTRODUCTION: Identification of hospitalized patients with suddenly unfavorable clinical course remains challenging. Models using objective data elements from the electronic health record may miss important sources of information available to nurses. METHODS: We recorded nurses' perception of patient potential for deterioration in 2 medical and 2 surgical adult hospital units using a 5-point score at the start of the shift (the Worry Factor [WF]), and any time a change or an increase was noted by the nurse. Cases were evaluated by three reviewers. Intensive care unit (ICU) transfers were also tracked. RESULTS: 31 159 patient-shifts were recorded for 3185 unique patients during 3551 hospitalizations, with 169 total outcome events. Out of 492 potential deterioration events identified, 380 (77%) were confirmed by reviewers as true deterioration events. Likelihood ratios for ICU transfer were 17.8 (15.2-20.9) in the 24 hours following a WF > 2, and 40.4 (27.1-60.1) following a WF > 3. Accuracy rates were significantly higher in nurses with over a year of experience (68% vs 79%, P = 0.04). The area under the receiver operator characteristic curve (AUROC) was 0.92 for the prediction of ICU transfer within 24 hours. DISCUSSION: This is a higher accuracy than most published early warning scores. CONCLUSION: Nurses' pattern recognition and sense of worry can provide important information for the detection of acute physiological deterioration and should be included in the electronic medical record.

Identifying patients at risk of inhospital death or hospice transfer for early goals of care discussions in a US referral center: the HELPS model derived from retrospective data.

OBJECTIVE: Create a score to identify patients at risk of death or hospice placement who may benefit from goals of care discussion earlier in the hospitalisation. DESIGN: Retrospective cohort study to develop a risk index using multivariable logistic regression. SETTING: Two tertiary care hospitals in Southeastern Minnesota. PARTICIPANTS: 92 879 adult general care admissions (50% male, average age 60 years). PRIMARY AND SECONDARY OUTCOME MEASURES: Our outcome measure was an aggregate of inhospital death or discharge to hospice. Predictor variables for the model encompassed comorbidities, nutrition status, functional status, demographics, fall risk, mental status, Charlson Comorbidity Index and acuity of illness on admission. Resuscitation status, race, geographic area of residence and marital status were added as covariates to account for confounding. RESULTS: Inhospital mortality and discharge to hospice were rare, with incidences of 1.2% and 0.8%, respectively. The Hospital End-of-Life Prognostic Score (HELPS) demonstrated good discrimination (C-statistic=0.866 in derivation set and 0.834 in validation set). The patients with the highest 5% of scores had an 8% risk of the outcome measure, relative risk 12.9 (10.9-15.4) when compared to the bottom 95%. CONCLUSIONS: HELPS is able to identify patients with a high risk of inhospital death or need for hospice at discharge. These patients may benefit from early goals of care discussions.

A stochastic model of acute-care decisions based on patient and provider heterogeneity.

The primary cause of preventable death in many hospitals is the failure to recognize and/or rescue patients from acute physiologic deterioration (APD). APD affects all hospitalized patients, potentially causing cardiac arrest and death. Identifying APD is difficult, and response timing is critical - delays in response represent a significant and modifiable patient safety issue. Hospitals have instituted rapid response systems or teams (RRT) to provide timely critical care for APD, with thresholds that trigger the involvement of critical care expertise. The National Early Warning Score (NEWS) was developed to define these thresholds. However, current triggers are inconsistent and ignore patient-specific factors. Further, acute care is delivered by providers with different clinical experience, resulting in quality-of-care variation. This article documents a semi-Markov decision process model of APD that incorporates patient and provider heterogeneity. The model allows for stochastically changing health states, while determining patient subpopulation-specific RRT-activation thresholds. The objective function minimizes the total time associated with patient deterioration and stabilization; and the relative values of nursing and RRT times can be modified. A case study from January 2011 to December 2012 identified six subpopulations. RRT activation was optimal for patients in "slightly concerning" health states (NEWS > 0) for all subpopulations, except surgical patients with low risk of deterioration for whom RRT was activated in "concerning" states (NEWS > 4). Clustering methods identified provider clusters considering RRT-activation preferences and estimation of stabilization-related resource needs. Providers with conservative resource estimates preferred waiting over activating RRT. This study provides simple practical rules for personalized acute care delivery.

Incidence and 1-Year Outcomes of Perioperative Atrial Arrhythmia in Elderly Adults After Hip Fracture Surgery.

OBJECTIVES: To determine the incidence and 1-year outcomes of an elderly population with perioperative atrial arrhythmia (PAA) within 7 days of hip fracture surgery. DESIGN: Retrospective cohort study. SETTING: The Rochester Epidemiology Project (REP). PARTICIPANTS: Elderly adults consecutive undergoing hip fracture repair from 1988 to 2002 in Olmsted County, Minnesota (N = 1,088, mean age 84.0 ± 7.4, 80.2% female). MEASUREMENTS: Baseline clinical variables were analyzed in relation to survival using Cox proportional hazards methods for comparison. RESULTS: Sixty-one participants (5.6%) developed PAA within the first 7 days. During 1 year of follow-up, 239 (22%) participants died. PAA was associated with greater mortality (45% vs 21%; hazard ratio (HR) = 2.8, 95% confidence interval (CI) = 1.9-4.2). Other mortality risk factors were male sex (HR = 2.0, 95% CI = 1.5-2.6), congestive heart failure (HR = 2.1, 95% CI = 1.7-2.8), chronic renal insufficiency (HR = 2.0, 95% CI = 1.5-2.8), dementia (HR = 2.9, 95% CI = 2.2-3.7), and American Society of Anesthesiologists risk Class III, IV, or V (HR = 3.3, 95% CI = 1.9-5.9). CONCLUSION: Elderly adults undergoing hip fracture surgery who develop PAA within 7 days have significantly higher 1-year mortality than those who do not. Further studies are indicated to determine whether prevention of PAA will reduce mortality in this population.

Why the C-statistic is not informative to evaluate early warning scores and what metrics to use.

Metrics typically used to report the performance of an early warning score (EWS), such as the area under the receiver operator characteristic curve or C-statistic, are not useful for pre-implementation analyses. Because physiological deterioration has an extremely low prevalence of 0.02 per patient-day, these metrics can be misleading. We discuss the statistical reasoning behind this statement and present a novel alternative metric more adequate to operationalize an EWS. We suggest that pre-implementation evaluation of EWSs should include at least two metrics: sensitivity; and either the positive predictive value, number needed to evaluate, or estimated rate of alerts. We also argue the importance of reporting each individual cutoff value.

Individualizing and optimizing the use of early warning scores in acute medical care for deteriorating hospitalized patients.

AIM: While early warning scores (EWS) have the potential to identify physiological deterioration in an acute care setting, the implementation of EWS in clinical practice has yet to be fully realized. The primary aim of this study is to identify optimal patient-centered rapid response team (RRT) activation rules using electronic medical records (EMR)-derived Markovian models. METHODS: The setting for the observational cohort study included 38,356 adult general floor patients hospitalized in 2011. The national early warning score (NEWS) was used to measure the patient health condition. Chi-square and Kruskal Wallis tests were used to identify statistically significant subpopulations as a function of the admission type (medical or surgical), frailty as measured by the Braden skin score, and history of prior clinical deterioration (RRT, cardiopulmonary arrest, or unscheduled ICU transfer). RESULTS: Statistical tests identified 12 statistically significant subpopulations which differed clinically, as measured by length of stay and time to re-admission (P < .001). The Chi-square test of independence results showed a dependency structure between subsequent states in the embedded Markov chains (P < .001). The SMDP models identified two sets of subpopulation-specific RRT activation rules for each statistically unique subpopulation. Clinical deterioration experience in prior hospitalizations did not change the RRT activation rules. The thresholds differed as a function of admission type and frailty. CONCLUSIONS: EWS were used to identify personalized thresholds for RRT activation for statistically significant Markovian patient subpopulations as a function of frailty and admission type. The full potential of EWS for personalizing acute care delivery is yet to be realized.

Widely used track and trigger scores: are they ready for automation in practice?

INTRODUCTION: Early Warning Scores (EWS) are widely used for early recognition of patient deterioration. Automated alarm/alerts have been recommended as a desirable characteristic for detection systems of patient deterioration. We undertook a comparative analysis of performance characteristics of common EWS methods to assess how they would function if automated. METHODS: We evaluated the most widely used EWS systems (MEWS, SEWS, GMEWS, Worthing, ViEWS and NEWS) and the Rapid Response Team (RRT) activation criteria in use in our institution. We compared their ability to predict the composite outcome of Resuscitation call, RRS activation or unplanned transfer to the ICU, in a time-dependent manner (3, 8, 12, 24 and 36 h after the observation) by determining the sensitivity, specificity and positive predictive values (PPV). We used a large vital signs database (6,948,689 unique time points) from 34,898 unique consecutive hospitalized patients. RESULTS: PPVs ranged from less than 0.01 (Worthing, 3 h) to 0.21 (GMEWS, 36 h). Sensitivity ranged from 0.07 (GMEWS, 3 h) to 0.75 (ViEWS, 36 h). Used in an automated fashion, these would correspond to 1040-215,020 false positive alerts per year. CONCLUSIONS: When the evaluation is performed in a time-sensitive manner, the most widely used weighted track-and-trigger scores do not offer good predictive capabilities for use as criteria for an automated alarm system. For the implementation of an automated alarm system, better criteria need to be developed and validated before implementation.

Evaluating implementation of a rapid response team: considering alternative outcome measures.

OBJECTIVE: Determine the prolonged effect of rapid response team (RRT) implementation on failure to rescue (FTR). DESIGN: Longitudinal study of institutional performance with control charts and Bayesian change point (BCP) analysis. SETTING: Two academic hospitals in Midwest, USA. PARTICIPANTS: All inpatients discharged between 1 September 2005 and 31 December 2010. INTERVENTION: Implementation of an RRT serving the Mayo Clinic Rochester system was phased in for all inpatient services beginning in September 2006 and was completed in February 2008. MAIN OUTCOME MEASURE: Modified version of the AHRQ FTR measure, which identifies hospital mortalities among medical and surgical patients with specified in-hospital complications. RESULTS: A decrease in FTR, as well as an increase in the unplanned ICU transfer rate, occurred in the second-year post-RRT implementation coinciding with an increase in RRT calls per month. No significant decreases were observed pre- and post-implementation for cardiopulmonary resuscitation events or overall mortality. A significant decrease in mortality among non-ICU discharges was identified by control charts, although this finding was not detected by BCP or pre- vs. post-analyses. CONCLUSIONS: Reduction in the FTR rate was associated with a substantial increase in the number of RRT calls. Effects of RRT may not be seen until RRT calls reach a sufficient threshold. FTR rate may be better at capturing the effect of RRT implementation than the rate of cardiac arrests. These results support prior reports that short-term studies may underestimate the impact of RRT systems, and support the need for ongoing monitoring and assessment of outcomes to facilitate best resource utilization.

Effect of obesity and clinical factors on pre-incision time: study of operating room workflow.

As the obese population is increasing rapidly worldwide, there is more interest to study the different aspects of obesity and its impact especially on healthcare outcomes and health related issues. Targeting non-surgical times in the operating room (OR), this study focuses on the effect of obesity along with clinical factors on pre-incision times in OR. Specifically, both the individual and combined effect of clinical factors with obesity on pre-incision times is studied. Results show that with the confidence of 95%, pre-incision time in the OR of obese patients is significantly higher than those for non-obese patients by approximately five percent. Findings also show that more complex cases do not exhibit significant differences between these patient subgroups.

Myocardial infarction after hip fracture repair: a population-based study.

OBJECTIVES: To quantify the occurrence of myocardial infarction (MI) occurring in the early postoperative period after surgical hip fracture repair and estimate the effect on 1-year mortality. DESIGN: A population-based, historical cohort study of individuals who underwent surgical repair of a hip fracture that used the computerized medical record linkage system of the Rochester Epidemiology Project. SETTING: Academic and community hospitals, outpatient offices, and nursing homes in Olmsted County, Minnesota. PARTICIPANTS: Over the 15-year study period (1988-2002), 1,116 elderly adults underwent surgical repair of a hip fracture. MEASUREMENTS: At the end of the first 7 days after hip fracture repair, participants were classified into one of three groups: clinically verified MI (cv-MI), subclinical myocardial ischemia, and no myocardial ischemia. One-year mortality was compared between these groups. Multivariate models assessed risk factors for early postoperative cv-MI and 1-year mortality. RESULTS: Within the first 7 days after hip fracture repair, 116 (10.4%) participants experienced cv-MI and 41 (3.7%) subclinical myocardial ischemia. Overall 1-year mortality was 22%, with no difference between those with subclinical myocardial ischemia and those with no myocardial ischemia. One-year mortality for those with cv-MI (35.8%) was significantly higher than for the other two groups. Occurrence of early postoperative cv-MI, male sex, and history of heart failure or dementia were independently associated with greater 1-year mortality, whereas prefracture home residence and preoperative higher hemoglobin were protective. CONCLUSION: Rates of early postoperative, cv-MI after hip fracture repair exceed rates after other major orthopedic surgeries and are independently associated with greater 1-year mortality.

Clinical presentation and outcome of perioperative myocardial infarction in the very elderly following hip fracture surgery.

BACKGROUND: Patterns of clinical symptoms and outcomes of perioperative myocardial infarction (PMI) in elderly patients after hip fracture repair surgery are not well defined. METHODS: A retrospective 1:2 case-control study in a cohort of 1212 elderly patients undergoing hip fracture surgery from 1988 to 2002 in Olmsted County, Minnesota. RESULTS: The mean age was 85.3 ± 7.4 years; 76% female. PMI occurred in 167 (13.8%) patients within 7 days, of which 153 (92%) occurred in first 48 hours; 75% of patients were asymptomatic. Among patients with PMI, in-hospital mortality was 14.4%, 30-day mortality was 29 (17.4%), and 1-year mortality was 66 (39.5%). PMI was associated with a higher inpatient mortality rate (odds ratio [OR], 15.1; confidence interval [CI], 4.6-48.8), 30-day mortality (hazard ratio [HR], 4.3; CI, 2.1-8.9), and 1-year mortality (HR, 1.9; CI, 1.4-2.7). CONCLUSION: Elderly patients, after hip fracture surgery, have a higher incidence of PMI and mortality than what guidelines indicate. The majority of elderly patients with PMI did not experience ischemic symptoms and required cardiac biomarkers for diagnosis. The results of our study support the measurement of troponin in postoperative elderly patients for the diagnosis of PMI, in order to implement in-hospital preventive strategies to reduce PMI-associated mortality.

Impact of heart failure on hip fracture outcomes: a population-based study.

BACKGROUND: Hip fracture and heart failure are becoming more prevalent conditions in hospitalized patients. Despite differences in postoperative outcomes from other intermediate risk procedures, guidelines classify hip fracture repair as an intermediate risk operation. OBJECTIVE: This population-based study sought to examine the prevalence and incidence of heart failure in hip fracture patients. DESIGN, SETTING, AND PATIENTS: We conducted a population-based historical cohort study of 1116 Olmsted County, MN residents undergoing 1212 hip surgeries from 1988 through 2002. Data were obtained through medical record review. Heart failure was defined by Framingham criteria. RESULTS: The prevalence of preoperative heart failure in our study population was 27% (327 of 1212 cases). Those with preoperative heart failure demonstrated longer lengths of stay, were more often discharged to a skilled facility, and had higher inpatient mortality rates. Rates of postoperative heart failure were 6.7% at seven days and 21.3% at one year. Postoperative heart failure was more common among those with preoperative heart failure (HR 3.0), and those with preoperative heart failure demonstrated higher postoperative mortality rates. Men had a higher risk of postoperative mortality compared to women. Overall survival was lowest among those with both preoperative and postoperative heart failure. CONCLUSIONS: Heart failure represents a common and serious perioperative condition in hip fracture patients. Hip fracture patients with and without heart failure carry higher postoperative risk than guidelines may suggest. Future work must focus on the perioperative management of hip fracture patients with and without heart failure to mitigate postoperative morbidity.

Unplanned transfers to the intensive care unit: the role of the shock index.

BACKGROUND: Unplanned (unexpected) transfers to the intensive care unit (ICU) are typically preceded by physiologic instability. However, trends toward instability may be subtle and not accurately reflected by changes in vital signs. The shock index (SI) (heart rate/systolic blood pressure as an indicator of left ventricular function, reference value of 0.54) may be a simple alternative means to predict clinical deterioration. OBJECTIVE: To assess the association of the SI with unplanned ICU transfers. DESIGN: Retrospective case-control study. SETTING: Academic medical center. PATIENTS: Fifty consecutive general medical patients with unplanned ICU transfers between 2003 and 2004 and 50 matched controls admitted to the same general medical unit between 2002 and 2004. MEASUREMENTS: Demographic data and vital signs abstracted from chart review. RESULTS: The SI was associated with unplanned ICU transfer at values of 0.85 or greater (P < 0.02; odds ratio, 3.0) and there was a significant difference between the median of worst shock indices of cases and controls (0.87 vs. 0.72; P < 0.005). There was no significant difference in age, race, admission ward, or Charlson Comorbidity Index, but hospital stay for cases was significantly longer (mean [standard deviation, SD], 14.8 [9.7] days vs. 5.7 [6.3] days; P < 0.001). CONCLUSIONS: SI is associated with unplanned transfers to the ICU from general medical units at values of 0.85 or greater. Future studies will determine whether SI is more accurate than simple vital signs as an indicator of clinical decline. If so, it may be a useful trigger to activate medical emergency or rapid response teams (RRTs).

Measuring hospital adverse events: assessing inter-rater reliability and trigger performance of the Global Trigger Tool.

OBJECTIVE: To determine the inter-rater reliability of the Institute for Healthcare Improvement's Global Trigger Tool (GTT) in a practice setting, and explore the value of individual triggers. DESIGN: Prospective assessment of application of the GTT to monthly random samples of hospitalized patients at four hospitals across three regions in the USA. SETTING: Mayo Clinic campuses are in Minnesota, Arizona and Florida. PARTICIPANTS: A total of 1138 non-pediatric inpatients from all units across the hospital. INTERVENTION: GTT was applied to randomly selected medical records with independent assessments of two registered nurses with a physician review for confirmation. MAIN OUTCOME MEASURE: The Cohen Kappa coefficient was used as a measure of inter-rater agreement. The positive predictive value was assessed for individual triggers. RESULTS: Good levels of reliability were obtained between independent nurse reviewers at the case-level for both the occurrence of any trigger and the identification of an adverse event. Nurse reviewer agreement for individual triggers was much more varied. Higher agreement appears to occur among triggers that are objective and consistently recorded in selected portions of the medical record. Individual triggers also varied on their yield to detect adverse events. Cases with adverse events had significantly more triggers identified (mean 4.7) than cases with no adverse events (mean 1.8). CONCLUSIONS: The trigger methodology appears to be a promising approach to the measurement of patient safety. However, automated processes could make the process more efficient in identifying adverse events and has a greater potential of improving care delivery and patient 'outcomes'.

Body mass index and the impact on hospital resource use in patients undergoing total knee arthroplasty.

We identified all total knee arthroplasty patients between 1996 and 2004 and classified them by preoperative body mass index (BMI) as normal (BMI, 18.5-24.9 kg/m(2)), overweight (BMI, 25.0-29.9 kg/m(2)), obese (30-34.9 kg/m(2)), or morbidly obese (≥ 35.0 kg/m(2)). Of 5521 patients, 769 had a normal BMI, 1938 were overweight, 1539 were obese, and 1275 were morbidly obese. Adjusted length of stay was no different between normal (4.85 days), overweight (4.84 days), obese (4.86 days), or morbidly obese patients (4.93 days) (P = .30). Overall costs were similar among normal ($15,386), overweight ($15,430), obese ($15,646), or morbidly obese patients ($15,752) (P = .24). Postsurgical costs were no different among normal ($9860), overweight ($9889), obese ($10,063), or morbidly obese patients ($10,136) (P = .44). Our results suggest that increased BMI does not lead to increased hospital resource use for total knee arthroplasty.

Body mass index (BMI) and risk of noncardiac postoperative medical complications in elderly hip fracture patients: a population-based study.

BACKGROUND: Obese patients are thought to be at higher risk of postoperative medical complications. We determined whether body mass index (BMI) is associated with postoperative in-hospital noncardiac complications following urgent hip fracture repair. METHODS: We conducted a population-based study of Olmsted County, Minnesota, residents operated on for hip fracture in 1988 to 2002. BMI was categorized as underweight (<18.5 kg/m2), normal (18.5-24.9 kg/m2), overweight (25.0-29.9 kg/m2), and obese (> or = 30 kg/m2). Postoperative inpatient noncardiac medical complications were assessed. Complication rates were estimated for each BMI category and overall rates were assessed using logistic regression modeling. RESULTS: There were 184 (15.6%) underweight, 640 (54.2%) normal, 251 (21.3%) overweight, and 105 (8.9%) obese hip fracture repairs (mean age, 84.2 +/- 7.5 years; 80% female). After adjustment, the risk of developing an inpatient noncardiac complication for each BMI category, compared to normal BMI, was: underweight (odds ratio [OR], 1.33; 95% confidence interval [CI], 0.95-1.88; P = 0.10), overweight (OR, 1.01; 95% CI, 0.74-1.38; P = 0.95), and obese (OR, 1.28; 95% CI, 0.82-1.98; P = 0.27). Multivariate analysis demonstrated that an ASA status of III-V vs. I-II (OR, 1.84; 95% CI, 1.25-2.71; P = 0.002), a history of chronic obstructive pulmonary disease (COPD) or asthma (OR, 1.58; 95% CI, 1.18-2.12; P = 0.002), male sex (OR, 1.49; 95% CI, 1.10-2.02; P = 0.01), and older age (OR, 1.05; 95% CI, 1.03-1.06; P < 0.001) contributed to an increased risk of developing a postoperative noncardiac inpatient complication. Underweight patients had higher in-hospital mortality rates than normal BMI patients (9.3 vs. 4.4%; P = 0.01). CONCLUSIONS: BMI has no significant influence on postoperative noncardiac medical complications in hip-fracture patients. These results attenuate concerns that obese or frail, underweight hip-fracture patients may be at higher risk postoperatively for inpatient complications.

A comparison of hospital adverse events identified by three widely used detection methods.

OBJECTIVE: Determine the degree of congruence between several measures of adverse events. DESIGN: Cross-sectional study to assess frequency and type of adverse events identified using a variety of methods. SETTING: Mayo Clinic Rochester hospitals. PARTICIPANTS: All inpatients discharged in 2005 (n = 60 599). INTERVENTIONS: Adverse events were identified through multiple methods: (i) Agency for Healthcare Research and Quality-defined patient safety indicators (PSIs) using ICD-9 diagnosis codes from administrative discharge abstracts, (ii) provider-reported events, and (iii) Institute for Healthcare Improvement Global Trigger Tool with physician confirmation. PSIs were adjusted to exclude patient conditions present at admission. MAIN OUTCOME MEASURE: Agreement of identification between methods. RESULTS: About 4% (2401) of hospital discharges had an adverse event identified by at least one method. Around 38% (922) of identified events were provider-reported events. Nearly 43% of provider-reported adverse events were skin integrity events, 23% medication events, 21% falls, 1.8% equipment events and 37% miscellaneous events. Patients with adverse events identified by one method were not usually identified using another method. Only 97 (6.2%) of hospitalizations with a PSI also had a provider-reported event and only 10.5% of provider-reported events had a PSI. CONCLUSIONS: Different detection methods identified different adverse events. Findings are consistent with studies that recommend combining approaches to measure patient safety for internal quality improvement. Potential reported adverse event inconsistencies, low association with documented harm and reporting differences across organizations, however, raise concerns about using these patient safety measures for public reporting and organizational performance comparison.